Win Prediction in Esports: Mixed-Rank Match Prediction in Multi-player Online Battle Arena Games

Esports has emerged as a popular genre for players as well as spectators, supporting a global entertainment industry. Esports analytics has evolved to address the requirement for data-driven feedback, and is focused on cyber-athlete evaluation, strategy and prediction. Towards the latter, previous work has used match data from a variety of player ranks from hobbyist to professional players. However, professional players have been shown to behave differently than lower ranked players. Given the comparatively limited supply of professional data, a key question is thus whether mixed-rank match datasets can be used to create data-driven models which predict winners in professional matches and provide a simple in-game statistic for viewers and broadcasters. Here we show that, although there is a slightly reduced accuracy, mixed-rank datasets can be used to predict the outcome of professional matches, with suitably optimized configurations

"It's Unwieldy and It Takes a Lot of Time." Challenges and Opportunities for Creating Agents in Commercial Games

Game agents such as opponents, non-player characters, and teammates are central to player experiences in many modern games. As the landscape of AI techniques used in the games industry evolves to adopt machine learning (ML) more widely, it is vital that the research community learn from the best practices cultivated within the industry over decades creating agents. However, although commercial game agent creation pipelines are more mature than those based on ML, opportunities for improvement still abound. As a foundation for shared progress identifying research opportunities between researchers and practitioners, we interviewed seventeen game agent creators from AAA studios, indie studios, and industrial research labs about the challenges they experienced with their professional workflows. Our study revealed several open challenges ranging from design to implementation and evaluation. We compare with literature from the research community that address the challenges identified and conclude by highlighting promising directions for future research supporting agent creation in the games industry.Comment: 7 pages, 3 figures, to be published in the 16th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE-20

Gamification design for motivating and measuring modal shift

Cities across the world attempt to minimise the negative environmental and wellbeing effects of increasing traffic volume and density. To this end, an increasing number of cities have taken to games and gamified applications to motivate mobility behaviours with less adverse effects. Being a novel approach predominantly deployed on online platforms, a major challenge of this approach is designing systems to generate valid in-the-wild mobility behaviour data to assess their effectiveness. Drawing on experiences from an on-going development project of a gamified application targeting tourist behaviour in York (UK) city centre, this paper discusses how a mobile gamified application driving sustainable behaviours can be designed to quantify its impact. It provides recommendations on how gamification design can allow for a measurable output on the levels of modal shift gained through in game promotion of alternative modes of transport

Exploration and Skill Acquisition in a Major Online Game

Using data from a major commercial online game, Destiny, we track the development of player skill across time. From over 20,000 player record we identify 3475 players who have played on 50 or more days. Our focus is on how variability in elements of play affect subsequent skill development. After validating the persistent influence of differences in initial performance between players, we test how practice spacing, social play, play mode variability and a direct measure of game-world exploration affect learning rate. These latter two factors do not affect learning rate. Players who space their practice more learn faster, in line with our expectations, whereas players who coordinate more with other players learn slower, which contradicts our initial hypothesis. We conclude that not all forms of practice variety expedite skill acquisition. Online game telemetry is a rich domain for exploring theories of optimal skill acquisition

Clyde: A deep reinforcement learning DOOM playing agent

In this paper we present the use of deep reinforcement learn-ing techniques in the context of playing partially observablemulti-agent 3D games. These techniques have traditionallybeen applied to fully observable 2D environments, or navigation tasks in 3D environments. We show the performanceof Clyde in comparison to other competitors within the con-text of the ViZDOOM competition that saw 9 bots competeagainst each other in DOOM death matches. Clyde managedto achieve 3rd place in the ViZDOOM competition held at theIEEE Conference on Computational Intelligence and Games2016. Clyde performed very well considering its relative sim-plicity and the fact that we deliberately avoided a high levelof customisation to keep the algorithm generic

Deterministic and Discriminative Imitation (D2-Imitation): Revisiting Adversarial Imitation for Sample Efficiency

Sample efficiency is crucial for imitation learning methods to be applicable in real-world applications. Many studies improve sample efficiency by extending adversarial imitation to be off-policy regardless of the fact that these off-policy extensions could either change the original objective or involve complicated optimization. We revisit the foundation of adversarial imitation and propose an off-policy sample efficient approach that requires no adversarial training or min-max optimization. Our formulation capitalizes on two key insights: (1) the similarity between the Bellman equation and the stationary state-action distribution equation allows us to derive a novel temporal difference (TD) learning approach; and (2) the use of a deterministic policy simplifies the TD learning. Combined, these insights yield a practical algorithm, Deterministic and Discriminative Imitation (D2-Imitation), which operates by first partitioning samples into two replay buffers and then learning a deterministic policy via off-policy reinforcement learning. Our empirical results show that D2-Imitation is effective in achieving good sample efficiency, outperforming several off-policy extension approaches of adversarial imitation on many control tasks.Comment: AAAI 202

Adaptive Scaffolding in Block-Based Programming via Synthesizing New Tasks as Pop Quizzes

Block-based programming environments are increasingly used to introduce computing concepts to beginners. However, novice students often struggle in these environments, given the conceptual and open-ended nature of programming tasks. To effectively support a student struggling to solve a given task, it is important to provide adaptive scaffolding that guides the student towards a solution. We introduce a scaffolding framework based on pop quizzes presented as multi-choice programming tasks. To automatically generate these pop quizzes, we propose a novel algorithm, PQuizSyn. More formally, given a reference task with a solution code and the student's current attempt, PQuizSyn synthesizes new tasks for pop quizzes with the following features: (a) Adaptive (i.e., individualized to the student's current attempt), (b) Comprehensible (i.e., easy to comprehend and solve), and (c) Concealing (i.e., do not reveal the solution code). Our algorithm synthesizes these tasks using techniques based on symbolic reasoning and graph-based code representations. We show that our algorithm can generate hundreds of pop quizzes for different student attempts on reference tasks from Hour of Code: Maze Challenge and Karel. We assess the quality of these pop quizzes through expert ratings using an evaluation rubric. Further, we have built an online platform for practicing block-based programming tasks empowered via pop quiz based feedback, and report results from an initial user study.Comment: Preprint. Accepted as a paper at the AIED'22 conferenc